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ABSTRACT 

This study evaluates symbols and formats for the F/A-18 
Hornet head-up display (HUD) and Attitude Directional 
indicator (ADI) for use by pilots in recovering from 
unusual aircraft attitudes. . Two surveys were conducted to 
collect pilot opinions on various symbols and formats, 
based on past experimental research and current 
recommendations. For the first survey, 60 F/A-18 pilots 
prioritized several types of HUD symbols according to the 
amount of information these symbols provide for the pilot 
while he is in an unusual attitude. In some cases, the 
pilots also were asked to choose their preferred symbols. 
The second survey was based on the results of the first and 
was administered to 56 F/A-18 pilots. These pilots 

selected their preferred HUD or ADI display formats, 
choosing one from two to five possibilities in each case. 
The specific symbols and formats that were evaluated are 
described in detail. Survey results are provided, and 

recommendations are made for display implementation and for 
further research and testing of symbols and formats. 
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INTRODUCTION 



I . 

A. BACKGROUND 

The Navy's F/A-18 Hornet aircraft represents a major 
step in the application of integrated controls and 

computer-controlled displays toward the reduction of pilot 
workload and enhancement of mission success (Figure 1). 
The Hornet crew station was designed to provide the 
capabilities of both the F-4 and A-7 aircraft. It is 
expected to perform both the fighter and attack roles in 
battle, and to be operable by one pilot. Mission 

reliability is increased by a combination of improved 
hardware reliability and functional redundancy. [Ref- l:p. 
82] 

The head-up display (HUD) is the F/A 18 primary flight 
instrument for weapon delivery and navigation, including 
manual and automatic carrier landing modes (Figure 2). The 
HUD is a display which projects collimated symbol imagery 
onto a glass surface in the pilot's forward field of view. 
The technique results in the head-up presentation of flight 
control information, possibly combined with weapon delivery 
cues. Yet it does not interfere with external visual cues 
from the scene normally viewed through the windshield. 

Attitude information, alphanumeric cues, and steering 
commands are projected on the HUD combiner. Pitch and roll 
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Figure 1. F/A-18 Main Instrument Panel [Ref. l:pp. 2-23] 
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HEADING POINTER 




HUD BASIC SYMBOLOGY 



Figure 2. F/A-18 Head-Up Display 
[Ref . 2 : p . 8 ] 

information is shown by a flight path ladder (also called a 
pitch ladder) and a bank scale. Airspeed and altitude are 
presented in a digital format. Heading is determined by use 
of a moving tape compass readout along the top of the 
display. Optics are focused at infinity for easy 
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assimilation by the pilot while scanning the area outside 
the crew station . [Ref . 2:p. 3] 

Rapid recognition of aircraft spatial orientation is 
essential for the pilot when recovering from unusual, 
unexpected attitudes during aerobatic and emergency 
maneuvers. Under these circumstances, recovery of the 
desired attitude using aircraft instruments is strongly 
affected by display design factors. During any loss of 
situational awareness by the pilot, the HUD may not provide 
sufficient spatial orientation cues for quick recovery. 
[Ref. 3:p. 14-1] 

Situational awareness encompases awareness of both the 
"big" (tactical) picture and also the "small" (spatial 
orientation) picture. Spatial orientation refers to the 
aircrew's awareness of: 

1. Aircraft attitude: pitch angle, pitch rate, bank 

angle, bank angle rate. 

2. Aircraft energy state, airspeed, acceleration. 

3. Proximity over terrain . [Ref . 4:p. 3] 

Spatial orientation is gained through cockpit displays, 
outside-the-cockpit visual cues, aural cues, and tactile 
cues. It may be lost through distraction, disorientation, 
misorientation , and g-induced loss of conciousness . 
Disorientation occurs when conflicting sensory inputs are 
perceived and conflicts are not resolved. The common 
example of this is vertigo caused by visual illusions 
during instrument flight. Misorientation occurs when 
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incorrect sensory inputs are perceived and treated as 
correct. A common example of this is the pilot who 
unknowingly flies off a frozen or badly processed gyro. 
[Ref. 4:p. 3] 

Since 1979, 14 F/A-18s have been lost in mishaps where 
loss of situational awareness, spatial disorientation, or 
unexplained flight into the surface are listed as confirmed 
or probable cause factors. This number represents 57% of 
all U.S. Navy and Marine Corps losses to date, plus, 50% of 
all Canadian and 100% of Royal Australian Air Force F/A-18 
losses. [Ref. 4:p. 2] 

Evidence points to inadequate or ambiguous attitude 
displays in the cockpit as a probable cause of situational 
awareness loss. A potential problem that has been 
identified is an inability of the pilot to recognize when 
he is in an unusual attitude and then recover while using 
the HUD and associated cockpit displays . [Ref . 4:p. 5] 

B. HUD SYMBOLOGIES AND FORMATS 

Of special concern for this study are HUD symbols and 
formats that may assist pilots in recovery from unusual 
attitudes. Some of the symbologies addressed here are 
currently in use on HUDs in operational Navy aircraft. 
Others have been proposed for use by various researchers 
who also have proposed modifications to the attitude 
direction indicator (ADI). Figures 3 to 8 illustrate 
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various HUD and ADI symbols and formats that are of 
interest. These are described below. 

1. Pitch Reference Frame 

The position and format of the pitch scale or 
ladder are cues that may provide information to the pilot 
when he is in an unusual attitude. The pitch reference 
frame consists of the ladder plus other symbols, 
representing fixed angles in space. These symbols are used 
as references for aircraft pitch and velocity vector 
symbols (see Figure 2). They include 

a. Horizon line: a horizontal line which 
represents zero pitch (the local horizontal) or the 
location of the real world horizon. 

b. Pitch ladder (also called flight path ladder) : 
a set of roughly parallel lines representing angular 
distances above and below the horizon, usually in 5-degree 
increments . 

c. Pitch ladder tails: short vertical lines that 
can be present on the ends of the above-horizon and below- 
horizon pitch ladders indicating the direction toward the 
horizon . 

d. Pitch ladder degree numerals: Numbers adjacent 
to the ladder tails representing angular distance above and 
below the horizon, in degrees. [Ref. 5:p. 4] 
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2. Fixed Aircraft Reference 

The fixed aircraft reference symbol (sometimes 
called Theta or a waterline symbol) represents an extension 
ahead of the aircraft of the fuselage reference line or 
other longitudinal aircraft reference line (see Figure 3) . 
The symbol usually is shaped like a "W" . It indicates 
relative pitch and roll ' angles of the aircraft when 
compared to the horizon (either artificial or real world) 
or to a displayed pitch ladder. [Ref. 5:p. 4] 




Figure 3. HUD With Waterline Symbol 
[Ref. 6:p.l5] 



3. Velocity Vector 

The velocity vector (sometimes referred to as Gamma 
or the flight path marker) is a symbol indicating the 
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linear projection of a vector representing the aircraft 
velocity (see Figure 4). It usually resembles an aircraft, 
as seen tail-on. The vector originates at the aircraft 
center-of-gravity or some other well-defined location on 
the aircraft. A location forward of the aircraft center- 
of-gravity is sometimes used as origin to provide pitch 
rate quickening of the velocity vector symbol. [Ref. 5:p. 
4] 

4. Air Mass Velocity Vector 

The air velocity vector is a symbol that represents 
the linear projection of a vector indicating aircraft 
velocity through the air mass (see Figure 5) . It resembles 
the lower half of the velocity vector. The inverse of this 
vector is the relative wind. [Ref. 5:p. 4] 




Figure 4. HUD With Velocity Vector Symbol 

[Ref. 6:p. ] 
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Figure 5. HUD With Air Velocity Vector 
5. Bank Indication 

The aircraft's bank angle is the angle between true 
vertical above the earth's surface and the plane defined by 
the aircraft’s vertical and longitudinal axes. The use of 
a bank index symbol on the HUD has been recommended for 
precision instrument flight. The index is an arrow 
pointing either at the sky or at the earth, attached to the 
velocity vector (see Figure 6). It is sometimes referred to 
as an Augie Arrow. The bank index reading must agree with 
that shown on head-down instruments . [Ref . 5:p. A2] 
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Figure 6. HUD With Augie Arrow 

6. Attitude Directional Indicator 

The conventional attitude directional indicator 
(ADI) is a primary flight reference instrument for a 
variety of civil and military aircraft. The ADI provides 
an artificial horizon format for use during instrument 
flight (see Figure 7) that allows the pilot to control the 
pitch and roll of the aircraft without visual reference to 
the earth's horizon. Color coding maybe used on the ADI to 
differentiate pitch attitude above and below the 

representation of the horizon. [Ref. 2:p. 2] 
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Figure 7. Standard Attitude Directional Indicator 

[Ref . 8 : p . 7 ] 



7. Advisory Labels and Legends 

Words on the HUD may give immediate information. An 
example includes descriptive words or commands such as 
"CLIMB" when the aircraft is at an extreme nose up position 
(see Figure 8). However, the actual words must be 
consistent with what the aircraft is actually doing. 

[Ref. 7:p. 3] 

8. Symbol Color 

HUD formats presently all are monochromatic. That 
is, green symbols are displayed on a clear background. Use 
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Figure 8. HUD With "CLIMB" Indicated 
[Ref. 5 : p . B7] 



of other colors on the HUD has been proposed, but presents 
problems . 

If colors are used on head-up formats they must be 
consistent in meaning with those used on head-down 
instruments. Each color used must be assessed for 
acceptable contrast against all likely background 
conditions. In a degraded or monochromatic mode, a color 
display must remain legible and unambiguous. Colors should 
only be used if an improvement over monochrome has been 
shown. [Ref. 7:p. 3] 
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C. RELATED STUDIES 

1. Head-Up Display Studies 

There has been concern over HUD symbology since the 
1970s when military pilots started using HUDs for routine 
instrument flight. In 1976 the Air Force Instrument Flight 
Center conducted a survey to determine the degree to which 
HUDs were used and problems that were encountered with 
them. The conclusions included concern over 

standardization of HUD formats and symbology when used as a 
primary flight reference. A pilot factors program was 
suggested to determine (1) whether HUDs are appropriate to 
use as a primary flight reference system and (2) what 
symbology and format are required for the HUD to be used as 
primary flight reference. [Ref. 9:p. 2] In a survey of 280 
pilots flying HUD-equipped airplanes, a tendency of pilots 
towards disorientation was reported [Ref. 10:p. 1] . 

A related study at the Naval Air Test Center (NATC) 
evaluated the utility of a workload assessment device to 
measure pilot workload for approach and landing tasks. The 
study revealed a trend towards more mental reserve capacity 
when pilots were flying while using a predominantly 
pictorial and symbolic HUD configuration, compared to 
conventional HUD formats with graphical scales and 
alphanumeric information. [Ref. 11 :p. 5] 

Problems with recovery from unusual attitudes when 
a HUD is used have been recognized for many years. Several 
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studies comparing various HUD formats and symbology have 
been carried out by human factors engineers. These studies 
have included variables related to cognitive processes 
involved in perception as well as to HUD symbology. 

Several military standards have been adopted 
related to HUD and other display symbology. Nonetheless, 
symbology variations continue to exist. The design of HUD 
pitch scales on the pitch ladder has been of major concern. 
HUD pitch scale symbols have been shown to have little 
pictorial realism based on studies at the Naval Air 
Development Center (NADC) [Ref. 8:p.3]. Redundant pitch 
cue combinations have been found to improve roll-recovery 
decision making. Horizon-pointing "tails" have been 
demonstrated to be superior to nadir-pointing "tails" on a 
pitch decision-making task [Ref. 3:p. 14-7]. 

To support standardization, four separate 
experiments were conducted with non-aircrew subjects 
evaluating pitch scale numerals, pitch bars, local versus 
global features (design of symbols attached to pitch bars), 
and roll and horizon interpretation [Ref. 3:p. 14-1]. The 
HUD formats and tasks studied were somewhat simpler than 
those actually used in aircraft. The formats presented 
only parts of the information normally displayed on HUDs; 
tasks performed by the operators were only a subset of 
pilot tasks, although they included a critical part of the 
operator's total HUD-related tasks. Pilot reactions to 
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unusual, unexpected attitudes were observed and evaluated 
in these studies. Experimental evidence from these studies 
supports adoption of the following HUD pitch scale design 
characteristics (see Figure 9): 

1. Continuous positive above-the-horizon pitch bars and 
broken (dashed) negative below-the-horizon pitch 
bars . 

2. Numerals on the sides of the pitch scales, above or 
below but not aligned with the pitch bar extremities. 

3. Negative signs for negative pitch scale numerals. 

4. Horizon-pointing tails and pitch bars sloping towards 
or away from the horizon, depending on attitude. 

A combination of horizon-pointing tails and horizon-sloping 

pitch bars may be the safest and most effective design 

solution. [Ref. 3:p. 14-8] 

In response to a questionnaire administered at 

NATC , one-third of the F/A-18 pilots queried gave below 

average ratings for the HUD as the primary attitude 

reference for quick interpretation of unusual attitudes 

[Ref. 12:p. 7], More specifically, six pilots reported 

that the pitch bars on the pitch ladder were difficult to 

interpret in nose-high and nose- low attitudes. Reasons 

for this include the rapid movement of the pitch bars and 

the use of dashed pitch bars below the horizon line that 

look very similar to the solid lines above the horizon. 

Further study was done by the Air Force in 1987 on 

recognition of and recovery from unusual attitudes. The 

experiment required pilots to recover from artificially 
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1. Continuous Positive 
Above-the-hor izon bars 

2. Broken negative below- 
the-horizon bars 

3. Numerals aligned above 
pitch bars extremities 

4. Negative sign for below- 
horizon scale 



5. 

6 . 




5. Horizon-pointing tails 

6. Horizon-pointing pitch 
bars 



7 . 



+ 



7. Nadir-pointing pitch 
bars 



Figure 9. Recommended HUD Pitch Scale 
Design Characteristics 
[Ref. 3 : p . 14-14] 



induced unusual attitudes using various HUD formats. All 
the symbologies were compared to baseline F/A-18 HUD 
symbology. A post-flight questionaire was also completed 
by each subject pilot, rating the display in terms of ease 
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of flying, ease of maintaining spatial orientation, and 
overall rating of the display [Ref. 7:p. 8] 

The results of this Air Force study indicate that a 
two-to-one pitch scale compression is advantageous. 
Additional bank information and an upward pointing cue 
(Augie Arrow) on the velocity vector are effective. 
Slanted pitch ladder bars pointing toward the horizon 
enhance recovery from unusual attitudes. Automatic 

deletion of the velocity vector symbol at high angles of 
attack also enhances recovery. Five composite symbologies 
were recommended for further evaluation: 

1. Composite I (Figure 10): 

a. F/A-18 style pitch ladder below the horizon and 
conventional pitch ladder above (pitch ladder 
modified to indicate heading when pitch exceeds 
+/-60 degrees; no pitch precession passing zenith 
or nadir ) . 

b. Compass heading on the horizon line with digital 
heading displayed above the waterline symbol. 

c. Bank index at the top of the display format free 
to move through 360 degrees (enhanced when bank 
exceeds +/-60 degrees). 

d. Automatic change to two-to-one pitch scaling and 
display of a roll arrow on the velocity vector 
symbol (or waterline) when pitch exceeds 30 
degrees or bank exceeds 60 degrees. 

e. Deletion of velocity vector automatically at high 
angle of attack (if Augie Arrow or sky arrow is 
being displayed, display waterline symbol). 

2. Composite II: 

a. Identical to Composite I except F/A-18 style 
pitch ladder with slanted pitch bars above and 
below horizon (pitch ladder modified to indicate 
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Figure 10. Composite Symbology I, 
When Aircraft is Inverted 
[Ref . 7 :p. 43] 
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heading when pitch exceeds +/- 60 degrees; no 
pitch precession passing zenith or nadir) . 

3. Composite III: 

a. Identical to Composite I except fulltime one-to- 
one pitch scaling. 

4. Composite IV: 

a. Identical to Composite I except fulltime two-to- 
one pitch scale compression. 

5. Composite V: 

a. Identical to Composite I except fulltime Augie 
Arrow. [Ref. 7:p. 39] 

2. Attitude Directional Indicator Studies 

Discussions with pilots, psychologists, and 

engineers at NATC in April 1985 indicated that the 
Attitude Directional Indicator (ADI) ball is the preferred 
choice for attitude indicator instrument [Ref. 2:p. 3]. 

The current ADI ball in the F/A-18 is small and poorly 
located (near the pilot's right knee). Using this 

information and results from the Royal Air Force 1984 study 
[Ref. 3], NADC compared the F/A-18 HUD pitch ladder to an 
ADI for the length of time required to recover from unusual 
attitudes. Two experiments were conducted. 

The first experiment, involving non-pilots, was a 
comparison of two pitch ladder formats and a pictorial 
representation of an ADI ball in a static display (see 
Figure 11) . The ability of these formats to aid the 
subject in deciding how to recover from unusual attitudes 
was assessed. 
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Figure 11. Static Display of Aircraft 
Attitude Directional Indicator Used in NADC Studies 

tRef . 8 : p . 8 ] 



The second experiment, involving both non-pilots 

and pilots, tested the format of the current F/A-18 pitch 

ladder (Figure 12) and an ADI ball in a medium-fidelity, 

ground-based simulator in the Crewstation Evaluation 

Facility at NADC. The static display ADI ball resulted in 

significantly faster decision times and was superior to the 

pitch ladder in aiding recovery from unusual attitudes. A 

revised pitch ladder (Figure 13), based on the four 

concepts noted above for improved HUD displays , did not 

surpass the current pitch ladder in decision times. 

Recommendations, based on the results of these two 
experiments would be to include an electromechanical 
ADI ball, in an optimal location, within the display 
suite and have it act as the primary instrument for 
attitude. If a pitch-ladder presented on a HUD is the 
primary flight instrument, as it is in the F/A-18, an 
ADI ball should be present as a secondary instrument in 
a location such that it would require very little eye 
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Figure 12. Current F/A-18 HUD Pitch Ladder Used in 
NADC Studies [Ref. 8:p. 8] 




Figure 13. Revised F/A-18 HUD Pitch Ladder Used in 
NADC Studies [Ref. 8:p. 8] 



translation to cross reference between the two 
displays. [Ref. 8:p. 23] 

The results of both experiments suggest that the inclusion 
of an ADI located within the central field of view would 
aid in unusual attitude recovery and improve pilot spatial 
orientation . 
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NADC has performed research to determine whether 
the addition of an electronically-generated ADI, displayed 
directly below the HUD, would aid pilots in recovery from 
unusual attitudes. The study compared three display formats 
for their ability to aid pilots in recovery from unusual 
attitudes. The first format was the graphic representation 
of an ADI (Figure 11). The second format was that used on 
the F/A-18 HUD (Figure 12) [Ref. 8:p. 6]. The third 

format was the concurrent use of the HUD and the ADI. The 

electronically generated ADI again resulted in 
significantly faster recovery times, when compared with the 
current F/A-18 HUD format. 

The reasons for this result might include: 

1. The superiority of color coding on the ADI for 

denoting sky and ground, versus solid and dashed 

pitch bars on the HUD. 

2. A slower, yet more controllable rate of movement of 
the pitch scale on the ADI, compared to the rapid 
movement of the HUD pitch ladder. 

Other advantages of the ADI format include an 

easily distinguishable horizon line and ease in obtaining a 
snapshot assessment of the aircraft's attitude [Ref. 2:p. 
26] . The concurrent use of the HUD and the ADI proved to 
be complementary during unusual attitude recovery. During 
recovery from extreme pitch attitudes, the strengths of 
each format compensated for the weaknesses in the other. 
The results of this study indicate that the addition of a 
centrally-located ADI display in the F/A-18 would improve 
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pilot performance during unusual attitude recovery. The 
ADI would allow pilots to conveniently crosscheck for 
attitude information displayed on the HUD. 

Kennedy has proposed an enhanced ADI display, as 
shown in Figure 14. This would display ADI symbology on 
one of the F/A-18's two Digital Display Indicator (DDI) 
multipurpose displays. The enchanced ADI combines 
available ADI and HUD data in a location close to the 
pilot's primary instrument scan. The display provides a 
realistic, pseudo three-dimensional format similar to the 
3-axis gyro used in flight training. [Ref. 4:p. 13] 

The enhanced ADI combines information from two 
head-down displays into a single format. It provides nose 
attitude, bank angle, heading, turn and slip needle, 
altitude, airspeed, angle of attack, load factor (g), 
maximum g over 4.0, and horizon-pointing arrows based on a 
Swedish design. The improved ADI is expected to be useful 
during normal operations as well as for night and 
instrument flying. 

D. OBJECTIVES OF STUDY 

The purpose of this study is to determine the HUD 
design variables and overall display format that best 
assist F/A-18 pilots in maintaining or rapidly regaining 
spatial orientation. 

Experimental studies have suggested various design 
variables and formats that assist decision-making 
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Figure 14. Enhanced Attitude Directional Indicator 

[Ref. 4] 

performance. A survey of F/A-18 pilots has been conducted 
to assess their opinions on these design variables and 
formats in a static environment, based on their experience. 
It is hypothesized that key design variables and formats 
can be identified as a result of the survey. These can be 
recommended for simulation testing and as performance-based 
evidence for HUD standardization. 

E. SCOPE 

This study is limited to those specific design 
variables and formats that previously have been 
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hypothesized to have significant effects on decision-making 
performance when a pilot must recover from an unusual 
aircraft attitude. Opinions have been collected on 
specific variables that aid in situational awareness, as 
supported by experimental study. Due to unavailability of 
facilities, experimental testing of the favored formats in 
a dynamic flight simulation by F/A-18 pilots has not been 
carried out for this study. 
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II. CONDUCT OF STUDY 



A. STUDY METHODOLOGY 

Two surveys were conducted for this study. The purpose 
of Survey 1 (Appendix A) was to isolate specific symbols 
that are present or could be present on the HUD format that 
may assist pilots in recovery from unusual attitudes. 
Survey 2 then was conducted to obtain pilot views on 
display formats incorporating the preferred symbols. 

1. Survey 1 

Survey 1 included questions on the pitch ladder 
tail formats and their location on the pitch ladder. The 
location is important since the pilot has time only for a 
quick look at the HUD symbols to determine his position 
while in an unusual attitude. 

The pitch ladder bars and how they might be angled 
were examined. The pitch ladder bars provide visual cues 
for determining the aircraft's angle of attack with respect 
to the horizon. 

A number is shown on each pitch bar to indicate the 
degrees above or below the horizon represented by that bar. 
The location and readability of the numbers are important 
due to the need to interpret them with only a quick look. 
Possible locations of the numbers were examined. 
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The usefulness of a negative sign adjacent to the 
below-horizon number also was examined. This additional 
visual cue may assist in determining attitude. 

Various types of velocity vector symbols and the 
presence of an Augie Arrow were included in the survey. 
Pilots were asked how helpful each of these would be during 
recovery from unusual attitudes. The usefulness of words 
that might be present on the HUD when the aircraft is in 
extreme attitudes (CLIMB or DIVE) was the final area 
examined . 

2. Survey 2 

The purpose of Survey 2 (Appendix B) was to 
incorporate the preferred symbols from the results of 
Survey 1 into overall HUD display formats. Questions 
regarding the format of the ADI also were included. 

The direction the Augie Arrow should point (toward 
the ground, horizon, or sky) was examined, because proper 
interpretation of the arrow during a quick look is 
necessary. The possible use of a contrasting color for the 
below-horizon pitch ladder symbols was examined. Color is 
a good visual cue, especially if the display is moving 
rapidly and the dashed lines become blurred. 

Pilot opinions were collected concerning 
presentation of word cues on the HUD while the aircraft is 
at an extreme angle. Word cues can be presented as commands 
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(CLIMB or DIVE) or as information (NOSE UP or NOSE DOWN) . A 
comparison of these types of descriptive wording was made. 

Pilots were asked their preference for retaining or 
deleting the velocity vector while at extreme angles of 
attack. This factor was examined because of the importance 
of including only those symbols on the HUD that the pilots 
think are necessary, and decluttering the format by 
removing the rest. 

Displaying the below-horizon pitch ladder as a 
"sawtooth" line has been proposed. Pilot opinions were 
solicited on this type of symbology and its usefullness as 
a visual cue in recovery from extreme attitudes. 

Five possible ADI formats also were examined. 
Research and informal discussion with pilots and scientists 
indicate that the ADI provides good visual cues for 
recovery from unusual attitudes and for routine flight if 
the HUD does not function properly. Pilots were asked to 
indicate which of the five formats they would prefer to 
have, in conjunction with a standard F/A-18 HUD format. 

B. PILOTS SURVEYED 

The pilots surveyed were all F/A-18 operational pilots 
from VFA-125 and VFA-113 squadrons at Lemoore Naval Air 
Station, California. Sixty pilots responded to the first 
questionaire . The average total flight hours for 
participants was 1000-2000 hours. The average tactical jet 
hours was 1000 hours. The average F/A-18 hours was less 
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than 500 hours. Four of the respondents had combat 
experience . 

Fifty-six pilots responded to the second questionaire . 
They averaged 1000-2000 total flight hours. The average 
tactical jet hours was 1000 hours. The average F/A-18 
hours was less than 500 hours. Four of the respondents 
had combat experience. 

C. SYMBOLOGIES AND FORMATS EVALUATED 

The symbologies and formats that were evaluated during 
this study are those that assist in maintaining situational 
awareness. That is, these symbols help the pilot recognize 
the status of his own aircraft and its relationship to the 
outside world. These are illustrated in Figure 3-8 and in 
Appendices A and B. 

D. SURVEY ADMINISTRATION 

1. Survey 1 

The first survey was administered at Lemoore Naval 
Air Station on two separate occasions but under similar 
conditions. The first administration was to 15 pilots in 
the VFA-113 squadron. The second was to VFA-125, with 45 
pilots responding. Appendix A provides the survey forms 
used . 

Each format was illustrated on the survey form and 
also was presented visually via transparancies projected 
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from an overhead projection machine. Participants were not 
allowed to discuss the specific formats or the questions. 

The first survey contained three questions 
regarding the pitch ladder tail formats, position of degree 
numerals, and velocity vector symbol cues. Pilots were 
required to rate the symbols, that is, to assign each of 
the illustrated instances to a ranked category, using 
categorical judgement. The descriptors that were associated 
with each category were related to the quality of 
information that each symbol possesses. These descriptions 
assisted the pilot with his task of rating those symbols. 

The other questions on the first survey asked the 
pilots to choose their favored format, when given two 
options. The specific formats illustrated two kinds of 
pitch ladder bar angles, presence or absence of a negative 
sign associated with below-horizon pitch bars, and use of 
words as visual cues. 

2. Survey 2 

The second survey was mailed to VFA-113 and VFA-125 
for administration by LCDR Dave Kennedy, Safety Officer, 
and LCDR Bob Woods, Training Officer, respectively. These 
officers were instructed to administer the survey in a 
fashion similar to that done with the first. The formats 
presented in the second survey were not presented visually 
from an overhead, but were illustrated in the survey forms 
(see Appendix B) . 
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The questions in the second survey asked the pilots 
to choose between two or three possible HUD formats. The 
survey included questions on (1) where an Augie Arrow 
should point (toward the ground, horizon, or sky) if it 
were present on the HUD format, (2) whether the below- 
horizon pitch ladder should be color coded, (3) what types 
of descriptive wording should be used when the aircraft is 
in an extreme angle of attack, (4) whether the velocity 
vector should be present at extreme angles of attack, and 
(5) whether further research should be done on an 
experimental below-horizon pitch ladder. 

The last question asked the pilots to choose 
between five ADI and HUD formats in combination. The ADI 
formats that were illustrated included the format that is 
currently used on the F/A-18 aircraft along with four other 
programmable options. 
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III. DATA ANALYSIS 



A. CATEGORICAL RATING ANALYSIS TECHNIQUE 

The data analysis method used to analyze survey results 
for questions 1, 3, and 5 in the first survey is a scaling 
method which uses categorical ratings provided by judges. 
The technique constructs an interval scale based on these 
categorical ratings. The interval scale includes not only 
the instances but also the bounds between the categories. 
In this case, instances are the specific HUD symbols that 
pilots were asked to rate; these make up the rows of the 
frequency array. The rating categories define the possible 
levels of information quality that the various symbols 
provide for the pilot. [Ref. 13] 

Five rating categories were used for this study: (1) 
Terrible Quality, (2) Poor Quality, (3) Fair Quality, (4) 
Good Quality, and (5) Excellent Quality. No assumptions are 
made about the relative interval sizes for the categories. 
The categories are understood to be a mutually exclusive 
set of successive intervals which collectively exhaust all 
possible responses. 

The ten-step procedure constructs an interval scale 
that includes the instances and bounds between categories 
[Ref. 14:p.l4]. The ten-step method requires several 
assumptions. The first assumption is that the pilot’s 
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judgements about the scale value of an instance i can be 
expressed as a normally distributed random variable with 
mean pi and variance 01 2 . 

The second assumption is that pilots view the continuum 
of values for instances as categories that are broken into 
successive intervals, each having an upper bound or 
boundary except the Excellent Quality category. The 
pilot's judgement about the category's upper bound is also 
expressed as a normally distributed random variable. 
Category j has a normally distributed upper bound with mean 
b 1 j and variance Vj 2 . 

The third assumption is that the pilot's judgements 
about the scale values of instances are stochastically 
independent random variables that have a correlation 
coefficient of zero for all pairs i and j. 

The fourth assumption is that all category bounds have 
the same variance, that is, Vj 2 = c for all j. [Ref. 14] 

B. TEN-STEP PROCEDURE FOR OBTAINING SCALE VALUES 

The ten-step procedure described below is taken from 
Reference 14. It is a method that yields scaled numerical 
data for the pilots' categorical responses concerning the 
HUD variable types. 

1. Arrange the raw frequency data in a table Fi j where 
the rows are instances i and the columns are 
categories j . The columns should be arranged in 
ascending order of category value, so that the last 
column to the right represents the most favorable 
category. 
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2. Compute relative cumulative frequencies for each row, 
and record these in a new table Pi j where Pi j is the 
proportion of pilots judging instance i in or below 
category j . The values in the right hand column of 
Pi j will equal one and that column may be omitted for 
computational purposes. 

3. Compute the Zi j array by treating the Pi j values as 
leftward areas under a Normal (0,1) curve and find 
the Z values for these areas in a table of values of 
the normal or Gaussian distribution. 

4. Compute the row average Zi for each row i in the Zi j 
array . 

5. Compute the column average bj for each column j in 
the Zi j array. The bj column averages are the upper 
bound values of category j on the scale. 

6. Compute the grand average b of all the values in the 
Zi j array. This is done by averaging the column 
averages bj . 

7. Compute the sum of squares for the column 
differences : 

m = l 

B = Z (bj - b} 2 . 

J = l 

8. Compute the sum of squares of the row differences: 

m = i 

Ai = Z (Zi j - Zi ) 2 . 

l = i 

9. Compute /(B/Ai ) for each row to give an estimate of 
/ (oi 2 +c) . 

10. Compute Si = b - Zi /(B/Ai ) for each row i. The Si 
values are the scale values of the instances, and are 
on the same interval scale as the category bounds b j . 
A linear transformation y = a + Bx , B>0 may be 
performed to move the scale where it is desired. The 
same transformation must be used to move the instance 
values and the category bounds. 
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C. OBTAINING SCALE VALUES FROM THE CATEGORICAL SURVEY DATA 
An example of the ten-step procedure for the pilot 
survey will be shown step by step for question 5 on Survey 
1. The scaling problem is broken into different problems 
because the Zi j array must be complete, as described in 
reference 13. 

1. The raw frequencies are given as illustrated in Table 
1. The categories T, P, F, G, and E represent 
terrible, poor, fair, good, and excellent quality of 
information for each variable. 

TABLE 1. FREQUENCY DATA FOR PROBLEM 5, Fi j . 

(See Appendix A for illustrations of the symbols identified 
here as A, B, C, D, E, and F.) 



Symbol 


T 




P 




F 


G 


E 


A 


4 




14 




7 


22 


ii 


B 


25 
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2 


0 


0 


C 


11 




27 




7 


9 


4 


D 


17 




29 




7 


6 


0 


E 


8 




20 




15 


14 


1 


F 


3 




7 




14 


24 


10 


2. The : 


relative 


cumulative frequencies 


are computed fc 


each 


row, as 


illustrated in 


Table 2 . 


The last colurr 


will 


always 


be a 


vector of 


ones and may be 


omitted . 


TABLE 


2. RELATIVE 


CUMULATIVE 


FREQUENCY 


DATA, 


Pi J 


Symbol 


T 






P 


F 




G 


A 


0.07 






0.31 


0.43 




0.81 


B 


0.43 






0.96 


1.00 






C 


0.19 






0.66 


0.78 




0.94 


D 


0.29 






0.79 


0 .88 






E 


0.14 






0.48 


1.00 






F 


0.05 






0.17 


0 . 41 




0.83 
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3 . 



The relative frequencies are then treated as leftward 
areas under a Normal (0,1) curve. The z values for 
the areas are recorded in Table 3. The values given 
in Table 2 are divided into two scaling problems 
because none of the pilots gave good or excellent 
ratings for three of the variables. 



TABLE 3. Z VALUES FOR THE NORMAL DISTRIBUTION 



Symbol 


T 


P 


F 


G 


A 


-1.48 


-0.50 


-0.17 


0.88 


C 


-0.88 


0.40 


0.76 


1.48 


F 


-1.63 


-0.94 


-0.22 


0.94 


B 


-0.18 


1.76 


3.90 




D 


-0.56 


0.81 


1.18 




E 


-1.08 


-0.05 


3.90 





4. The row averages, Zt , are computed, as shown in Table 
4. The column averages, bj , are also computed in 
Table 4. 



TABLE 4. ROW AND COLUMN AVERAGES 



Zi j 


T 


P 


F 


G 


Zi 


A 


-1.48 


-0.50 


-0.17 


0.88 


-0.32 


C 


-0.88 


0.40 


0.76 


1.48 


0.44 


F 


-1.63 


-0.94 


-0.22 


0.94 


-0.46 


bj 


-1.33 


-0.35 


0.12 


1.10 




B 


-0.18 


1.76 


3.90 




1.83 


D 


-0.56 


0.81 


1.18 




0 . 48 


E 


-1.08 


-0.05 


3.90 




0.92 


bj 


-0.60 


2.52 


2.99 
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5. The grand average, b, is computed. For this example, 
that calculation is: 

b AC F = (-1.33 + -0.346 + 0.123 + l.l)/4 = -0.113 
buDE = (-0.60 + 2.52 + 2 . 99 ) / 3 = 1.64. 

6. The sum of squares of the column averages, B, is 
calculated: 



m=l _ 

B = Z (bj - b) 2 
3 = 1 

Bacf = (-1.33 0.113) 2 + (-0.349 - -0.113) 2 

+ (0.123 - -0.113)2 = 3.059 
Bbde = (-0.60 - 1.64)2 + (2.52 - 1.64) 2 
+ (2.99 - 1.64)2 = 7.61. 



7 . 



The sum of squares of the row averages is calculated 
for each row of the Zi j array. 





m=l 






E (Zij - 
3=1 


Zi ) 2 


Aa 


= (-1.48 - 


-0.318)2 + 




+ (-0.17 


- -0.318)2 


Ac 


= (-0.88 - 


0.44)2 + ( 




0.44)2 + 


(1.48 - 0. 


Af 


= (-1.63 - 


-0.463)2 + 




+ (0.123 


- -0.463)2 


Ab 


= (-0.18 - 


1.83)2 + ( 




+ (3.9 - 


1.83)2 = 8 


Ad 


= (-0.56 - 


0.48 ) 2 + 




+ (1.18 - 


■ 0.48)2 = 


Ae 


= (-1.08 - 


0.92)2 + ( 




+ (3.9 - 


0.92)2 = 1 



(-0.50 - -0.318)2 
+ (0.88 - -0.318)2 = 2.84 

0.40 - 0.44)2 + (0.76 - 
44)2 = 2.926 

(-0.346 - -0.463)2 
+ (1.1 - -0.463)2 = 4.15 

1.76 - 1.83)2 
.32 

(0.81 - 0.48)2 

1.68 

-0.05 - 0.92)2 
3.82. 



8. The value of /(B/Ai ) is calculated for each row: 



Aa = (3.059/2.84) • 3 =1.04 
Ac = (3.059/2.926) • 3 =1.02 
Af = (3.059/4.157)- 3 = 0.86 
Ab = (7.61/8.32) • 3 =0.96 
Ad = (7.61/1.68) • 3 = 2.13 
Ae = (7.61/13.82) • 3 =0.74. 

9. The scale values of the variable types are given for 
each row by the formula: 
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Si = b - Zi / (B/Ai ) 

The values for the Sis are as follows: 



Sa = -0.113 - (-0.318) (1.04) = 0.217 
S B = -0.113 -(0.44) (1.02) = -0.56 
Sr = -0.113 -(-0.463) (0.86) = 0.285 

Sb = 1.64 - (1.83) (0.96) = -0.117 

So = 1.64 - (0.48) (2.13) = 1.617 

S E = 1.64 - (0.92) (0.74) = 0.952. 



10. A linear transformation with the equation Y = a + (5x, 
is then done to put the two sets of values for 
question 5 on the same scale. Transformed results 
are : 



Sb = -1.638 + (-0.507) (-0.117) = -1.579 
Sd = -1.638 + (-0.507) (-0.8198) = -2.45 
Se = -1.638 + (-0.507) (0.952) = -2.12. 



The ten-step procedure for scaling categorical 
data outlined above was applied to questions 1, 3, 
and 5 in Survey 1. The columns of the raw frequency 
data array with values of zero had to be grouped with 
adjacent columns so that the Zi j array would not be 
incomplete. The scale values are found in Appendix C 
and in Figures 15, 16, and 17. 



D. SURVEY 1 DATA ANALYSIS 

The method described above was used to analyze the 
results from questions 1, 3, and 5 in Survey 1. 

The remaining questions (2, 4, 6, and 7) in Survey 1 

required the pilots to choose between two format variables. 
Such an "either/or" choice is representive of a discrete 
distribution. This distribution is hypothesized to be a 
Binomial Distribution in this case. If there is no 
preference for one or the other, it is hypothesized that, 
for each pilot, there is a 0.5 chance he will choose a 
given format variable. It follows that 
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Pr(X = x) = ( n ) p x q n - x . [Ref. 15:p. 100] 

Results are included in Appendix C. 

E. SURVEY 2 DATA ANALYSIS 

All the questions in Survey 2 required the pilots to 
choose one from among two, three, or five format variables. 
Again, it is hypothesized that the responses have a 
Binomial Distribution. That is, unless pilots have a 
preference, there is a 0.5 chance of selection for each 
when there are two formats, a 0.33 chance for three 
formats, and a 0.2 chance for each with five formats. It 
follows that 

Pr (X = x) = ( n ) p x q n “ x . [Ref. 15:p. 100] 

X 

Results are included in Appendix D. 
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IV. RESULTS 



A. SURVEY 1 RESULTS 

Data from the first survey were analyzed according to 
the procedure outlined in the previous chapter, and are 
available in Appendix C. The results for each question are 
discussed in detail below. 

1. Pitch Ladder "Tail" Formats 

Question 1: Paying attention only to the HUD pitch 
ladder tail formats, rate each of the following tail 
positions according to the quality of information and cues 
they would give you during recovery from unusual attitude. 

The first question asked the pilots to scale the 
five pitch tail formats on a rating scale. The rating 
categories, as described in the data analysis section, 
define the possible levels of information quality that the 
various symbols provide the pilot. The pilots rated each of 
five symbols. 

The highest-rated pitch ladder tail format is that 
which places both tails (above the horizon and below the 
horizon) at the outer ends of the pitch ladder, pointing 
towards each other (see Figure 14). Analysis placed this 
symbol in the Good category (see Figure 15). A total of 34 
of the pilots placed this symbol in the Good or Excellent 
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r 






1. Pitch ladder with tails 
at outer ends, pointing 
toward each other. 



2. Bars are level at 
horizon, with 
angle increasing 
with increased 
distance from 
horizon . 






5 



S 



5 



3. Degree numbers placed at 
both outer ends of both 
horizon bars. 



4. No negative sign 
present on 
be low-horizon 
pitch bars. 



Figure 14. Symbols Favored on the First Survey 
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5. Velocity Vector with 
Augie Arrow. 



5. Velocity Vector 
without Arrow. 




6. Word "CLIMB" 



7. Word "DIVE" 



present . 



present . 



Figure 14. Symbols Favored on the First Survey (Continued) 
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Levels of Quality 



4J 




Figure 15. 



Scaling Results for 



the First Survey, 



Question 1 
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category. All other symbols rated only Fair or Poor. Other 
data values can be found in Appendix C. 

Pilot comments included support for minimal clutter on 
the HUD, such as "Do not want too much clutter in the 
middle of the HUD". 

2. Pitch Ladder Bar Angles 

Question 2: Paying attention only to the angle of 
the pitch bars, circle the format that would give you the 
better quality of information during recovery from unusual 
attitude . 

The second question had the pilots choose their 
favored format when given two illustrated options. The 
preferred option (53 of the pilots) showed the pitch ladder 
bars level at the horizon, with the angle of the bars 
increasing with increased distance of the aircraft from the 
horizon (see Figure 14) . The probabilities were determined 
by using the statistical package Minitab. The solution is 
limited to four significant figures. The probability of at 
least 53 out of 59 pilots choosing this format, given pilot 
indifference, is 0.0000. 

3. Position of Degree Numerals 

Question 3: Paying attention only to the position 
of the numbers on the pitch ladder that represent degrees 
above and below the horizon, rate each of the following 
numeral positions according to the quality of information 
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and cues they would give you during recovery from unusual 
attitude . 

This question asked the pilots to rate 18 different 
numeral positions on the pitch ladder. The same rating 
categories were used as in question one. 

The highest-rated format included numerals shown at 
both outer ends of both the above the horizon and below the 
horizon pitch ladder (see Figure 14). Analysis placed this 
symbol in the Good category (see Figure 16) . A total of 45 
of the pilots placed this symbol in the Good or Excellent 
category. All other symbols rated only Fair or Poor. 

Pilot comments on this question support the concept 
of less clutter in the center of the HUD. These included, 
"Don't want too much clutter in the middle of HUD" and "My 
eyes are focused to the outer ends for the "tails" and it 
is easier to see the numbers when they are located there". 

4. Negative Sign 

Question 4: Paying attention to the below-horizon 
numbers on the pitch bars, circle the format which would 
give you the most information during recovery from unusual 
attitudes . 

Pilots were asked to choose whether a negative sign 
should be displayed adjacent to the below-horizon number 
representing degrees. The option without the negative sign 
was preferred by 35 of the respondents (see Figure 14). 
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Levels of Quail 



4J 




Figure 16. 



Scaling 



Results for the First Survey 



Question 3 
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The probability of at least 35 out of 60 pilots choosing 
this format, given indifference, is 0.0775. 

Pilot comments regarding the negative sign included 
"Too much clutter" and "The dashed line of the below- 
horizon bar is enough without the negative sign". One 
respondent also noted that "The use of a two color display 
would be helpful". 

5. Velocity Vector Symbol Cues 

Question 5: At high angles of attack, rate the 
following symbols according to the quality of information 
and cues they would give you during recovery from unusual 
attitudes . 

The fifth question asked the pilots to rate five 
different velocity vector symbols, with and without an 
Augie Arrow. The same rating categories were used as in 
questions one and three. 

The standard velocity vector or flight path marker, 
with and without an Augie Arrow, rated the highest of the 
symbols (see Figure 14). Analysis placed both the velocity 
vector with the Augie Arrow and the velocity vector without 
the Augie Arrow in the Good category (see Figure 17). A 
total of 11 pilots placed the velocity vector with the 
Augie Arrow in the Excellent category, while 22 rated it in 
the Good category. Without the Augie Arrow, ten placed the 
symbol in the Excellent category and 24 in the Good 
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Figure 17. Scaling Results for the First Survey 

Question 5 
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category . 



Other analysis results can be found in Appendix 



C. 

Pilot comments indicated concern as to whether an 
arrow would be informative or directive. Comments included, 
"Format needs to be directive? both arrows and words should 
indicate which way to recover", and "In low altitude, an 
arrow should show where the ground is". 

6 . CLIMB As a Cue 

Question 6: Circle the format which would give you 
the most information to make an efficient recovery, when 
your aircraft is at an unusual attitude. 

For this question, pilots chose their favored 
format when given two options: the presence of the word 
"CLIMB" or no word on the HUD. Pilots choosing the format 
with the word "CLIMB" present totalled 34 (see Figure 14) . 
The probability of at least 34 pilots out of 60 choosing 
this format, given indifference, is 0.1225. 

Pilot comments indicate that a directive word would 
be helpful. They also noted that the format should remain 
unchanged, using the zenith and nadir symbols already 
present on the current HUD. 

7. DIVE As A Cue 

Question 7: Circle the format which would give you 
the most information to make an efficient recovery, when 
your aircraft is at an unusual attitude. 



49 



The seventh question asked the pilots to choose 
whether the word "DIVE" should be present on the HUD, when 
appropriate. A total of 37 of the pilots preferred using 
the word "DIVE” (see Figure 14). The probability of at 
least 37 pilots out of 60 choosing this format, given 
indifference, is 0.0295. 

B. SECOND SURVEY RESULTS 

Data from the second survey were analyzed according to 
the procedure outlined in the previous chapter. Summary 
results are given in Figure 18. The data for each question 
are available in Appendix D. The results for each question 
are discussed in detail below. 

1. Augie Arrow Direction 

Question 1: If an arrow is present on the HUD 
format as displayed, what should it point to? Circle one. 

The first question had the pilots choose which 
direction (ground, horizon, or sky) an Augie Arrow should 
point if it were present on the HUD. Forty-three out of 56 
pilots chose the inclusion of an arrow. The "SKY" option 
was preferred by 23 of the 43 pilots (see Figure 18). The 
probability of at least 23 pilots out of 43 choosing this 
format, given indifference, is 0.0017. 

It should be noted that 13 of the pilots indicated 
that they do not want any arrow and chose none of the 
formats. Several comments on the surveys support the 
concept that an arrow may not be a good idea: "May make 
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1. Augie Arrow pointed 
toward the sky. 



2. Use of Color 




3. Word "CLIMB" used 



4. Word "DIVE" used 



Figure 18. Formats Favored on the Second Survey 



51 





5. Velocity Vector with 
Augie Arrow. 



6. "Sawtooth" Pitch 
Ladder . 




7. Enhanced ADI with Standard HUD. 



Figure 18. Formats Favored on the Second Survey (Continued) 
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the velocity vector harder to interpret (especially 
inverted) " and "The arrow may get confused with TACAN 
navigation" . 

Comments supporting the use of an arrow included, 
"The arrow needs to be directive for recovery", and "In a 
spin you always place the stick in the direction of the 
arrow. Following the same line of thinking, in the unusual 
attitude you would want to know where the sky is so as to 
pull that direction" . 

2 . Color on HUD 

Question 2: If the below-horizon pitch ladder were 
shown in a contrasting color to all other HUD symbology, 
e.g., red, would that assist in recovery from an unusual 
attitude? 

A total of 39 of the pilots responded affirmatively 
to this question (see Figure 18). The probability of at 
least 39 out of 56 pilots choosing this format, given 
indifference, is 0.0009 

The comments regarding the use of color included, 
"Use color that doesn't ruin night vision". The fact that 
color "would assist in unusual attitude recovery but may 
clutter up the HUD with more unnecessary information for a 
regular situation" also was noted. 
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3. 



Informational Cues When Nose is Up 



Question 3: At extreme angles of attack, if words 
are used, what words would you prefer to be shown? Circle 
one . 

Pilots were asked to choose between the words 
"CLIMB" and "NOSE UP" as informative cues describing the 
position of the aircraft- Thirty-eight out of 56 pilots 
chose the presence of words on the HUD. The word "CLIMB" 
was preferred by 26 out of the 38 (see Figure 18). The 
probability of at least 26 pilots out of 38 choosing this 
format, given indifference, is 0.0069. 

A total of 18 of the pilots do not want any words, 
and did not choose either word. Several of the comments on 
the surveys support not adding a word cue on the HUD: 
"Prefer no words", "Not a good idea even though marked 
"CLIMB", and "Not sure words would help". 

4. Informational Cues When Nose is Down 

Question 4: At extreme angles of attack, if words 
are used, what words would you prefer to be shown? Circle 
one . 

The fourth question asked the pilots to choose 
between the words "DIVE" and "NOSE DOWN" as informative 
cues describing the position of the aircraft. Thirty-seven 
out of 56 pilots chose the presence of words on the HUD. 
The word "DIVE" was preferred by 23 out of the 37 pilots 
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(see Figure 18). The probability of at least 23 pilots out 
of 37 choosing this format, given indifference, is 0.0494. 

It should be noted that 19 of the pilots do not 
want any words and did not choose either word. Comments 
included, "Not a good idea, words take too much attention 
demand", "Prefer no words", and "Unnecessary clutter with 
words" . 

5. Velocity Vector as a Cue 

Question 5: At extreme angles of attack, greater 
than +60 degrees or less than -60 degrees, what format do 
you like best? Circle one. 

The options were (1) having the velocity vector 
present, along with an Augie Arrow and the waterline 
symbol, or, (2) deleting the velocity vector and displaying 
only the Augie Arrow on the waterline symbol. A total of 
45 of the respondents preferred retaining the velocity 
vector (see Figure 18). The probability of at least 45 
pilots out of 56 choosing this format, given indifference, 
is 0 . 0000 . 

Pilot comments expressed concern with HUD clutter 
if many symbols are present. However, as one comment 
indicated, "The velocity vector with an arrow is a strong 
visual cue". 

6. Experimental Pitch Ladder 

Question 6: It has been proposed that the below- 
horizon pitch ladder be displayed as a "sawtooth" line. 
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Should this display format be further tested to determine 
if it would assist in recovery from unusual attitudes? 
Circle one. 

Approval for further experimental study was given 
by 32 of the respondents (see Figure 18). The probability 
of at least 32 pilots out of 56 choosing this format, given 
indifference, is 0.1144. 

7 . ADI Formats 

Question 7: Several ADI formats have been proposed 
for use on the F/A-18 DDI , with a standard HUD format 
present. Of the following HUD/ADI format combinations, 
circle the one you most prefer. 

Pilots were asked to choose the one they prefer 
from five different HUD/ADI combinations. The ADI format 
illustrated in Figure 18 was strongly preferred (47 of the 
pilots). The probability of at least 47 pilots out of 56 
choosing this format, given indifference, is 0.0000. 
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V. CONCLUSIONS AND RECOMMENDATIONS 



A. CONCLUSIONS 

The results of surveys carried out for this study show 
pilot support for the symbols currently used on the F/A-18 
HUD. In order to provide additional visual cues while the 
aircraft is in an unusual attitude, pilots show moderate 
support for the use of the words CLIMB or DIVE (57%-62%), 
an Augie Arrow (57%) , and color to designate below-horizon 
angles (70%) . 

Previous research on the use of words has investigated 
the use of "informative" words (such as NOSE UP or NOSE 
DOWN) to cue the pilot that he is in an extreme situation. 
A large proportion prefer that no words be used (33%-34%). 
If such cues are provided, more pilots prefer the use of 
"directive" words such as CLIMB or DIVE (41%-46%) to the 
previously proposed NOSE UP/NOSE DOWN terms (21%-25%) . 

The survey showed limited support for addition of an 
Augie Arrow symbol (57%) . Preferred direction for the 
arrow to point is towards the sky (41%). Pilot comments 
indicate that an arrow would be most helpful in a directive 
role. The arrow should be allowed to be part of the 
declutter option, however, so that the large proportion of 
pilots who will not use it can remove the symbol from the 
HUD. 
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The survey showed strong support for the use of color 
on the HUD (70%). Concern over possible interference with 
night vision was expressed, however. Further research on 
use of specific unusual attitude recovery symbols on the 
HUD is supported by a majority of pilots (57%). The 
"sawtooth" below-horizon pitch ladder is expected to 
provide stronger visual cues to indicate that the aircraft 
is in an unusual attitude. 

The strongest support for a crew station change is for 
a new format for the ADI when displayed on a DDI . A total 
of 84% of respondents prefer an ADI (used in combination 
with standard F/A-18 HUD formats) that gives a strong 
visual indication of above- and below-horizon angles. The 
ability to observe such an ADI format during normal 
instrument scan is deemed very important. 

The use of a survey of military pilots has been 
informative for isolating areas for further research. It 
is apparent that the pilots surveyed are satisfied with the 
current F/A-18 HUD formats and symbols. It has been 
observed that the loss of aircraft due to loss of 
situational awareness may be due to training problems. 
Training command aircraft do not use the same HUD formats 
as do tactical aircraft. Thus pilots may not receive 
enough experience in HUD use prior to assignment to an F/A- 
18 squadron. 
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B. RECOMMENDATIONS 



It is recommended that the symbols and formats 
currently used on the F/A-18 HUD be retained. In addition, 
an Augie Arrow should be an option for the pilots to use if 
there is a possibility of being in an unusual attitude. 
Since color is a good visual cue (especially when the HUD 
symbols are moving rapidly) , a color cue should be 
considered as a below-horizon indicator on the HUD and the 
ADI . 

The pilots showed the strongest support for a change in 
the ADI format. The pilots comments and responses indicate 
that the HUD is satisfactory but the ADI display is a more 
useful visual cue in unusual attitudes. The enhanced ADI, 
as proposed, should be incorporated in the crew station as 
soon as possible. 

Further experimental research should be carried out in 
the following areas: 

1. The use of directive versus informative words when 
the pilot is in extreme situations. 

2. The use of a sky pointer Augie Arrow. 

3. The use of color on the HUD and ADI. 

4. New formats for the ADI when displayed on the DDI . 

5. Visually distinct pitch ladders. 

All new formats should be tested dynamically in a 
ground-based simu]lator similar to the Crew Evaluation 
Facility at NADC . The dependent variables should include 
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pilot reaction time and time to recover from an unusual 
attitude for each of the independent variables. 
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APPENDIX A. FIRST SURVEY FORM 



INTRODUCTION 

Background: Several F/A-18s have been lost in mishaps 

where loss of situational awareness, spatial 

disorientation, or unexplained flight into the surface are 
listed as confirmed or probable cause factors. Evidence 
points to inadequate or ambiguous attitude displays 
available to the pilot. A potential problem may be an 
inability of the pilot to recognize when he is in an 
unusual attitude and then recover using the HUD. 

Objective of this study: The purpose of this 

questionaire is to determine what types of variables 
present on the HUD can best assist the pilot 1) to 
determine his current position during an unusual attitude 
and 2) to make an efficient recovery. 

Please respond to the questions taking into account 
your experience while flying aircraft and using HUDs . I am 
interested in your recommendations for the HUD variables 
that would best assist you in unusual attitude situations. 
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F/A 18 Aircrew Survey on HUD Variables 
During Unusual Attitudes 



Total Hours 
Tac Jet Hours 
F/A 18 Hours 

Combat Experi 
Service 



( ) < 500 ( ) 500 - 1000 

( ) 1000 - 2000 ( ) > 2000 

( ) < 500 ( ) 500 - 1000 

( ) 1000 - 2000 ( ) > 2000 

( ) < 500 ( ) 500 - 1000 

( ) 1000 - 2000 ( ) > 2000 

ence ( ) YES ( ) NO 

( ) USN ( ) USMC 
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HUD FORMAT CUES DURING UNUSUAL ATTITUDE 



PITCH LADDER "TAIL" FORMATS 

1. Paying attention only to the HUD pitch ladder "tail" 
formats, rate each of the following tail positions 
according to the quality of information and cues they would 
give you during recovery from unusual attitude. 



Quality of Information During Recovery From Unusual Attitude 



A. 


Terrible 

Quality 


Poor 

Quality 


Fair 

Quality 


Good 

Quality 


Excellent 

Quality 
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( ) 


( ) 


( ) 
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PITCH LADDER BAR ANGLES 



2. Paying attention only to the angle of the pitch bars, 
circle the format that would give you the better quality of 
information during recovery from unusual attitude. 




Constant 

Continuous 



Angle 

Sloping 



Bars are Level at Horizon, 
with Angle Increasing with 
Increased Distance from 
Horizon 
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POSITION OF DEGREE NUMERALS 



3. Paying attention only to the position of the numbers on 
the pitch ladder that represent degrees above and below the 
horizon, rate each of the following numeral positions 
according to the quality of information and cues they would 
give you during recovery from unusual attitude. 



Quality of Information During Recovery From Unusual Attitude 

Terrible Poor Fair Good Excellent 

Quality Quality Quality Quality Quality 



* 



A., 



3 




i ) 



( ) 



( ) i ) 



B. 

. i 

( ) 



* 



( ) 



( ) 



( ) ( ) 



C. 



( ) 



( ) 



( ) 



( ) 



( ) 



D. 



, 1 

( ) 

$ * 



( ) ( ) ( ) ( ) 



E. 



( ) i ) ( ) ( ) ( ) 
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Quality of Information During Recovery From Unusual Attitude 



Terrible Poor Fair Good 

Quality Quality Quality Quality 



3 



F. 



s 



( ) 



( ) 



( ) 



( ) 



G. 

~ ) ( ) ( ) ( ) 



H. 

i 

a ( ) ( ) ( ) ( ) 



I. 

~ ( ) ( ) ( ) ( ) 



J. 

i j 



3 



( ) ( ) ( ) ( ) 



K. 



( ) ( ) ( ) ( ) 

« 



L. 

( ) ( ) ( ) ( ) 

f T 



Excellent 

Quality 
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( ) 



( ) 



( ) 



( ) 
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Quality of Information During Recovery From Unusual Attitude 



Terrible Poor Fair Good 

Quality Quality Quality Quality 

M. 

1 i 

( ) ( ) ( ) ( ) 

* — * 



N. 



T" 

( ) ( ) ( ) ( ) 



0 . 

T 5 

. . ( ) i ) ( ) i ) 



P. 

T 

( ) ( ) ( ) < ) 



0 . 

t r 

( ) ( ) ( ) ( ) 



R. 

J 5 



( ) ( ) ( ) ( ) 

5 3 



Excellent 

Quality 



( ) 



( ) 



( ) 



( ) 



( ) 



< ) 
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NEGATIVE SIGN 



4. Paying attention to the below-horizon numbers on the 
pitch bars, circle the format which would give you the most 
information during recovery from unusual attitudes. 



5 



5 



5 
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VELOCITY VECTOR SYMBOL CUES 



5. At high angles of attack, rate the following symbols 
according to the quality of information and cues they would 
give you during recovery from unusual attitudes. 



Quality of Information During Recovery From Unusual Attitude 

Terrible Poor Fair Good Excellent 

Quality Quality Quality Quality Quality 




D. 
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WORDS AS CUES 



6. Circle the format which would give you the most 
information to make an efficient recovery, when your aircraft 
is at an unusual attitude. 



S It | 


1 s « 


*o L_ 






I s ® 








C l 1 


1 M 0 




05 J 


1 05 


05 1 






■ - ---1GS 



7. Circle the format which would give you the most 
information to make an efficient recovery, when your aircraft 
is at an unusual attitude. 
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APPENDIX B. SECOND SURVEY FORM 



Introduction and Instructions 

Background: Several F/A 18s have been lost in mishaps 
where loss of situational awareness, spatial disorientation, 
or unexplained flight into the surface are listed as 
confirmed or probable cause. Evidence points to inadequate or 
ambiguous attitude displays available to the pilot. A 
potential problem may be an inability of the pilot to 
recognize when he is in an unusual attitude and then recover 
using the HUD. 

Objective of this study: I am attempting to isolate HUD 
and other display variables that pilots prefer or recommend 
for use in recovery from unusual aircraft attitudes. I am 
most interested in determining what cues can best help the 
pilot, in order to prevent further mishaps. Results of this 
questionaire will be used for further experimental study in a 
realistic environment. 

This is the second of two questionaires eliciting 
opinions on the best way to display information related to 
unusual aircraft attitudes. In the first (which you may have 
filled out), opinions were obtained on the preferred knids of 
symbols for HUD displays. This second questionaire takes into 
account the results of that first survey; the most-preferred 
symbols now have been incorporated into overall HUD display 
formats. Possible formats for an ADI also are included here. 

Please respond to the questions, taking into account your 
experience while flying aircraft and using the HUD and the 
ADI. I am interested in your recomendations for the HUD 
variables, alone and in combination with the ADI, that would 
best assist you in unusual attitude situations. 
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F/A 18 Aircrew Survey on HUD Variables 
During Unusual Attitudes 



Total Hours 


( 


) 


< 500 


( 


) 


500 - 


1000 




( 


) 


1000 - 2000 


( 


) 


> 2000 




Tac Jet Hours 


( 


) 


< 500 


( 


) 


500 - 


1000 




( 


) 


1000 - 2000 


( 


) 


> 2000 




F/A 18 Hours 


( 


) 


< 500 


( 


) 


500 - 


1000 




( 


) 


1000 - 2000 


( 


) 


> 2000 




Combat Experi 


ence 




( ) YES 


( 


) 


NO 




Service 






( ) USN 


( 


) 


USMC 
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AUGIE ARROW DIRECTION 



1. If an arrow is present on the HUD format as displayed, 
what should it point to? Circle one. 
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GROUND 


HORIZON 




Comments : 
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COLOR ON HUD 



2. If the below-horizon pitch ladder were shown in a 

contrasting color to all other HUD symbology e.g. red, 
would that assist in recovery from an unusual attitude 
Circle one. 




YES 



NO 



Comments : 



WORDING OF INFORMATIONAL CUES 



3. At extreme angles of attack, if words are used, what 
words would you prefer to be shown? Circle one. 





Comments : 
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WORDING OF INFORMATIONAL CUES 



4. At extreme angles of attack, if words are used, what 
words would you prefer to be shown? Circle one. 




Comments : 



VELOCITY VECTOR AS A CUE 



5. At extreme angles of attack, greater than +60 degrees 
or less than -60 degrees, what format do you like best 
Circle one. 




velocity vector present 




delete velocity vector 



Comments : 



EXPERIMENTAL PITCH LADDER 



6. It has been proposed that the below-hor izon pitch 

ladder be displayed as a "sawtooth" line. Should this 
display format be further tested to determine if it 
would assist in recovery from unusual attitudes? 
Circle one. 




YES 



NO 



Comments : 
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ADI FORMATS 



7. Several ADI formats have been proposed for use on 
DDI, with a standard HUD format present. Of the 
following HUD/ADI format combinations, circle the 
you most prefer. Continues on following page. 





the 

one 
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APPENDIX C. FIRST SURVEY RESULTS 



HUD FORMAT CUES DURING UNUSUAL ATTITUDE 
PITCH LADDER "TAIL" FORMATS 

1. Paying attention only to the HUD pitch ladder "tail" 
formats, rate each of the following tail positions according 
to the quality of information and cues they would give you 
during recovery from unusual attitude. 



Quality of Information During Recovery From Unusual Attitude 
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Quality 


Quality 
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PITCH LADDER BAR ANGLES 



2. Paying attention only to the angle of the pitch bars, 
circle the format that would give you the better quality of 
information during recovery from unusual attitude. 




Constant Angle 
Continuous Sloping 



Bars are Level at Horizon, 
with Angle Increasing with 
Increased Distance from 
Horizon 



Number 6 



53 



11 % 



89% 
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POSITION OF DEGREE NUMERALS 



3. Paying attention only to the position of the numbers on 
the pitch ladder that represent degrees above and below the 
horizon, rate each of the following numeral positions 
according to the quality of information and cues they would 
give you during recovery from unusual attitude. 



Quality of Information During Recovery From Unusual Attitude 







Terrible 

Quality 


Poor 

Quality 


Fair 

Quality 


Good 

Quality 


Excellent 

Quality 
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Quality of 


Information During Recovery From 


Unusual 


Attitude 
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Quality of 


Information During Recovery From 


Unusual 


Attitude 
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NEGATIVE SIGN 



4. Paying attention to the below-horizon numbers on the 
pitch bars, circle the format which would give you the most 
information during recovery from unusual attitudes. 



j 



5 



Number 25 



35 



42% 



58% 
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VELOCITY VECTOR SYMBOL CUES 



5. At high angles of attack, rate the following symbols 
according to the quality of information and cues they would 
give you during recovery from unusual attitudes. 



Quality of Information During Recovery From Unusual Attitude 







Terrible 


Poor 


Fair 


Good 


Excellent 


Quality 


Quality 


Quality 


Quality 


Quality 


Ruaber 4 


14 


7 


22 


11 


7% 


24% 


12% 


38% 


19% 


Scale 

Value 0.22 










Ruaber 25 


31 


2 


0 


0 


43% 


54% 


3% 


0% 


0% 


Scale 

Value -1.58 










Ruaber 11 


27 


7 


9 


4 


19% 


47% 


12% 


15% 


7% 


Scale 

Value -0.56 










Ruaber 17 


29 


7 


6 


0 


29% 


49% 


12% 


10% 


0% 


Scale 

Value -2.45 










Ruaber 8 


20 


15 


14 


1 


14% 


35% 


25% 


24% 


2% 


Scale 

Value -2.12 










Ruaber 3 


7 


14 


24 


10 


5% 


12% 


24% 


42% 


17% 


Scale 

Value 0.29 











87 



WORDS AS CUES 



6. Circle the format which would give you the most 
information to make an efficient recovery, when your aircraft 
is at an unusual attitude. 



SC 1 


lie 


SO , 






i sb 
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Number 


26 


34 




43% 


57% 



7. Circle the format which would give you the most 
information to make an efficient recovery, when your aircraft 
is at an unusual attitude. 




Number 


37 


23 




62% 


38% 
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APPENDIX D. SECOND SURVEY RESULTS 



AUGIE ARROW DIRECTION 

1. If an arrow is present on the HUD format as displayed, 
what should it point to? Circle one. 



r~ 



i 



I »» I 



a 4t 
M 04 
0 1 0 
T.O 







GROUND 



HORIZON 



Number 2 



Number 18 




32% 



SKY 



Number 23 
41% 



No Arrow 
Number 13 
23% 
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COLOR ON HUD 



2. If the below-horizon pitch ladder were shown in a 

contrasting color to all other HUD symbology e.g. red, 
would that assist in recovery from an unusual attitude? 
Circle one. 




YES NO 



Number 


39 


17 




70% 


30% 
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WORDING OF INFORMATIONAL CUES 



3. At extreme angles of attack, if words are used, what 
words would you prefer to be shown? Circle one. 




46% 




12 

21 % 



No Words 
Number 18 
33% 
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WORDING OF INFORMATIONAL CUES 



4. At extreme angles of attack, if words are used, what 
words would you prefer to be shown? Circle one. 



ISO 009 B!9 1 1 

V t 1 1 1 l 1 * 1 

! A • 


J JSO 008 Its 1 

I 1 % 1 l 1 1 1 

! 


j 13351 

is-.- 1 i 


1 

Oil] . „ fTTi^q ] 


i . . . > o\ 1 

“ : / 


! c.- - 

1 / 


L — - 

/ N \ 


l / \ , 


/ \ 


L lit' ' J. 


k_ . A 



Number 23 


14 


41% 


25% 



No Words 

Number 19 
34% 
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VELOCITY VECTOR AS A CUE 



5. At extreme angles of attack, greater than +60 degrees 
or less than -60 degrees, what format do you like best? 
Circle one. 




velocity vector present 

Number 45 
80% 




delete velocity vector 

11 

20 % 
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EXPERIMENTAL PITCH LADDER 



6. It has been proposed that the below-hor izon pitch 

ladder be displayed as a "sawtooth" line. Should this 
display format be further tested to determine if it 
would assist in recovery from unusual attitudes? 
Circle one. 




YES 


NO 


Number 32 


24 


57% 


43% 
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ADI FORMATS 



7. Several ADI formats have been proposed for use on the 
DDI, with a standard HUD format present. Of the 
following HUD/ADI format combinations, circle the one 
you most prefer. Continues on following page. 
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7 
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